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ABSTRACT 

Electronic troubleshooting expert ise was explored in 
the three- contexts (design, production, and repair) that reflect 
distinct problem solving task environments* The purpose of the effort 
was to provide a more precise definition of the boundaries of 
expertise in electronicr- troubleshooting and the relationship of 
context to the development of troubleshooting instruction* Actual 
troubleshooting performance was studied in contextually 
representative tasks. Two subjects (engineers and technicians) were 
selected from each of the following areas: (1) design engineering; 
(2) production testing; and (3) customer service or field service* 
Troubleshooting on an aircraft electrical system simulator board with 
design, production, or normal wear flaws was studied* Beyond knowing 
where to start in the process, actual troubleshooting process 
differences were few. Generally, all subjects were able to perform in 
a manner consistent with their expertise, and there appeared to be 
little overall difference \n the ability of the troubleshooters to 
generate and evaluate hypotheses, acquire and interpret information, 
or select an appropriate problem space beyond the contextual 
reference* Difficulties were generally overcome through 
self-correction and continued data gathering. Subjects from all 
groups were able to demonstrate high levels of skill on these 
problems that were representative of typical job contexts. 
Implications for instruction in troubleshooting are explored. Five 
figures and five tables illustrate the discussion. (Author/SLD) 
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Abstract 

The purpose of this study was to explore electronic troubleshooting expertise in three 
contexts (design, production, and repair) that reflect distinct problem solving task environments, in 
an effort to provide a more precise definition of the boundaries of expertise in electronics 
troubleshooting and the relationship of context to the development of troubleshooting instruction. 
Based on the cognitive science approach to the study of expertise, the investigation explored actual 
troubleshooting perfcrrjance in contextually representative tasks. Although broad generalizations 
are limited due to sample size, important implications for the study of electronics troubleshooting 
expertise were discovered. 

The contexts under study represented three distinct task environments, defined by job 
related responsibilities. It appeared that beyond knowing where to start in the process, actual 
troubleshooting process differences were few. Generally, all subjects were able to perform in a 
manner consistent with expertise as defined by solving the problem representative of their 
respective contexts. There appeared to be little overall difference in the ability of the 
troubleshooters to generate and evaluate hypotheses, acquire and interpret information, or select an 
appropriate problem spaces beyond the contextual reference. Difficulties with interpretation of 
information or hypothesis evaluation were generally overcome through self correction and 
continued data gathering. In essence subjects from all groups were able to demonstrate high levels 
of skill associated with expertise in electronics troubleshooting within problems representative of 
their typical job contexts and experience. 
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Context and Expertise: The Case of Electronic Troubleshooting 

The cognitive science approach to the study of expertise holds promise for improved 
troubleshooting instruction (Bedard, & Frederiksen, 1992; Foshay, 1989; Fredericksen, 1984; 
Johnson, 1987; Johnson, Flesher, Ferej, & Jehng, 1992; Keller, 1985), However, research in 
electronic troubleshooting has focused primarily on maintenance scenarios (Johnson, 1989; Keller, 
1985; Lesgold et al, 1986; Rasmussen & Jensen, 1974), or has been guided by singular experts 
(Bedard & Frederiksen, 1992; White & Frederiksen, 1987). The organization of domain 
knowledge and its typical application appears to depend upon the tasks that experts routinely 
perform (Smith, 1990). In electronics troubleshooting, Rasmussen and Jensen (1974) reported 
differences in problem solving behavior between design engineers and maintenance technicians. 
The problem solving goal of maintenance technicians is the discovery of a fault in a system that has 
been working properly. Design engineers, howe\ er, are more likely to consider the theoretical 
constructs and basic conceptual operation of a faulty system. The purpose of this study was to 
explore electronic troubleshooting expertise in three typical problem solving contexts (design, 
production, and repair) in an effort to provide a more precise definition of the boundaries of 
expertise in electronics troubleshooting and the relationship of contextual emphasis to the 
development of troubleshooting instruction. 

The Role of Context 

Context is consistently recognized as an important element in problem solving research 
(Ericsson & Smith, 1991; Johnson, 1988; Keller, 1985; Stepich, 1991). However, the focus of 
expertise research has been on problem solving processes and theories of potential cognitive 
structure and function, and not on the effects of external environments or stimuli (Tennyson, 
1990). The role of context is implicit in the cognitive science approach to the study of expertise. A 
problem solver brings an internal set of values, knowledge, and skill to each environmentally 
bound problem situation. In effect an internal context absorbs and interacts with the external 
context resulting in observable behavior. The term domain itself is certainly a reflection of a 
specific environmental situation. The discovery of domain-specific pattern recognition in chess by 
Chase and Simon (1973), and the numc.^us validating studies in diverse domains, may in fact 
imply that superior performance is tied to a contextual reference, or structure. The chess board and 
the rules of the game may represent a context for problem solving that cues appropriate domain- 
related knowledge including a set of specific global and move related goals, rule structures, and 
past memory of effective strategies and styles of previous opponents. 

Cognitive competence is developed and demonstrated in specific external contexts. This 
contextual structure determines the domain of relevance (Hagendorf, 1990). Studies of expert 
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performance have repeatedly found superior performance to be linked to specific domains, or in 
effect, situations (Chase and Simon, 1973; Chi, Feltovich, & Glaser, 1981; Egan & Schwartz, 
1979). Context influences the perception and methods used to solve problems. Initially, process 
is linked to a particular domain, acquired in response to an environmental challenge faced by the 
developing organism. Context directs attention to certain problems, and particular aspects within 
problems, playing a crucial role in cognition. Context also leads to the selection of resources in the 
problem solving process (Ceci & Nightingale, 1990). 

Context has been found to exhibit important effects in many cognitive domains. These 
include visual perception (Ceci & Nightengale, 199C; Neisser, 1976) speech recognition (Massaro 
& Cohen, 1991) and semantic similarity (Miller & Charles, 1991). Lawrence (1988) studied the 
differences between novices and experts injudicial decision making. The expert judges were 
found to have specific "frames of reference" concerning case types which guided their decision 
making process. These frames are representational schemas that describe procedural operations in 
combination with preexistent perspectives. These perspectives appear to influence the objective or 
goal assigned by the judges to each type of case. 

In a study of expertise in classical genetics, Smith (1990) concluded that there can be more 
than one kind of expertise within a discipline based on the typical manner in which experts apply 
knowledge in response to specific task demands. Three groups, biology faculty members, genetic 
counselors, and undergraduate college students were given genetics problems to solve and were 
instructed to organize the problems by how they would be solved and then to circle the key words 
that supported the organization. Although faculty and counselors were both found to be successful 
at solving the problems the faculty members tended to organize the problems based on concepts 
while both the counselors and students focused on the problem knowns and unknowns. This 
difference in knowledge organization among the two expert groups appears to reflect the different 
purposes for which that knowledge is applied. 

Knowledge organization may also be different for electronics troubleshooting experts based 
on typical applications. Rasmussen and Jensen (1974) report that their collected protocols clearly 
indicate that maintenance technicians define their task as the search for a fault in a system which 
has previously operated properly. The designer, however, defines the task as understanding the 
basic function of the system, comparing conceptualizations to observation. Johnson (1987) 
discovered differences in expert behavior in an investigation of novice and expert troubleshooting. 
In that study two of the experts took considerably longer to solve a problem than did the novices 
because they got sidetracked by what they perceived as a design flaw. It seems plausible that the 
experts' slower performance may have been a result of a design oriented frame of reference. 
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Contextual problem solving continuum . In order to study the relationship of context in 
problem solving, a method must be established to define that element (Tennyson, 1992). In most 
research on expertise, or problem solving, the context has been implied by the rarely defined 
"domain" reference of the study (i.e., technical troubleshooting, physics problem solving, or 
medical diagnosis). Certainly in any of these cases, the generalizability of results must be limited 
by the goal structure imposed by the research design, the specific task set, and the individuals 
selected to participate in the study. 

Two common attributes of problem solving behavior are found extensively in the literature; 
that problem solving is goal based (Anderson, 1987; Bereiter, 1990; Cooke, 1988; Garner, 1990, 
Glaser, 1985; Hagendorf, 1990; Johnson 1988; Simon, 1981; Tennyson, 1992) and problem 
solving is reflective of the environment in which it occurs (Bereiter, 1990, Simon, 1981, Garner, 
1990; Johnson, 1988; Tennyson, 1992; Thomas & Litowitz, 1986). A model that represents the 
contextual element of a problem solving activity must therefore be based on these two elements. 



Insert Figure 1 about here 



Any effort to determine context free universal problem definitions based on characteristic 
attributes beyond physical or relational concepts are dependent upon the problem solver and 
contextual presentation. A system related definitional approach removes the inherently individual 
response to the amount of problem structure and definition. Accordingly, the problem of 
developing a framework for contextual evaluation has two additional requirements; the need to 
allow for completely individual responses and situations, and a definition based on physical or 
externally related concepts. Through an extension of the product/process life cycle paradigm 
(Collins & Devanna, 1990; Hayes, Wheelwright & Clark, 1988) a model was developed based on 
the complete life cycle of a single process or product from initial concept through discardment. 
This model, called the Contextual Problem Solving Continuum (see Figure 1) has three discrete 
stages (a) design or development, (b) implementation or production, and (c) maintenance or repair. 
Each phase has a distinct goal related structure. Problems in the design phase are novel 
presentations requiring the development of new knowledge and structure. Problems in the 
implementation phase require application of a known process or production of a viable design. 
Problems in the maintenance phase require the continuation of a specified standard, through 
adherence to a conceptual structure, or pre-determined physical condition. The individual 
relevance of thi? model is a function of environment and goal structure. The domain frame 
provides the environment (i.e., electronics, physics, or medicine). The individual's current 
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problem solving goal determines the overall relationship of that person to a phase in the model 
(i.e., repair a previously working generator set, build a structure from blueprints, or design an 
electronic amplifier with unity gain). 

Technical Troubleshooting Model 

Through a synthesis of research studies, Johnson (1989) developed a Technical 
Troubleshooting Model that reflects the cognitive process flow of an individual engaged in 
troubleshooting a technical problem. The model is divided into two main phases (a) hypothesis 
generation and (b) hypothesis evaluation. In phase one the problem solver acquires information 
from internal or external sources that can be used to support a representation of the problem. 
Following the creation of this representation one or more hypotheses are developed that may 
account for the fault In phase two, the problem solver evaluates a hypothesis that was generated 
in phase one and attempts to confirm or dis-confirm the potential cause of failure (Johnson, 1989). 

In the first phase of the Technical Troubleshooting Model, the troubleshooter attempts to 
identify possible faults through a series of information acquisition efforts. These efforts enable the 
troubleshooter to develop a mental representation of the potential area in which the fault could exist 
called a problem space. The troubleshooter may obtain information from both internal and external 
sources. The internal sources available to the troubleshooter include declarative and procedural 
knowledge stored in the individual's long term memory (Johnson, 1989). Tennyson (1990) adds 
contextual knowledge which includes embedded situational criteria governing selection of 
appropriate knowledge structures. External information acquisition sources include job aids, 
technical support, technical evaluation utilizing test procedures or operational adjustments, and 
sensory evaluation through visual, tactile, auditory, and olfactory inspection (Johnson, 1989). 
These sources comprise a "toolbox" of available support for the troubleshooter. The quality of this 
toolbox will be dependent upon the resources available and the declarative and procedural 
knowledge bases of the troubleshooter. Keller (1985) notes that in realistic situations a 
troubleshooter may only have a sub-set of these resources available or in working order. 

The second phase of the Technical Troubleshooting model involves the evaluation of a 
generated hypothesis. This process involves obtaining additional information to support a decision 
to either accept or reject the proposed hypothesis (Frederiksen, 1984). During this phase the 
troubleshooter determines if the potential fault is the true fault, and if able to make such a 
determination can then repair the problem. Troubleshooting is often a cyclic process, with each 
level representing a closer approximation of the actual fault (White & Frederiksen, 1987; 
Rasmussen & Jensen, 1974). If the fault is not identified in this phase, then the troubleshooter 
repeats foe process of hypothesis generation (Johnson, 1989). 
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The process of troubleshooting requires an integrated ability to collect, process, and 
evaluate external and internal information. A correct fault solution may be obtained after many 
instances of incorrect hypothesis evaluation, or information interpretation. Conversely an incorrect 
judgment may result from a single failure to correctly interpret acquired information or evaluate 
generated hypothesis. This implies that an important element in the process is the ability to 
continue the cycle of processes represented in the Technical Troubleshooting Model, and to verify 
proposed solutions (Johnson et al., 1992). 

Method 

The purpose of this investigation was to compare technical troubleshooting performance in 
the three context areas. The investigation examined actual troubleshooting behavior on a set of 
contextually representative tasks. Data to support the investigation was primarily in the form of 
verbal protocols collected during the troubleshooting activity. The protocol data was supported by 
researcher observations and interviews conducted immediately following that activity. 
Additionally, graphic representations of each subject's area of responsibility, area of expertise, and 
problem space representations for the troubleshooting task were collected using the Contextual 
Problem Solving GonuiiUinr. as a guide. 

Subjects 

The subjects selected for this study were professional engineers and technicians employed 
at Frasca International, Incorporated. Frasca is a manufacturer of a full line of state of the art 
aircraft flight simulators and radio products. Design, manufacturing, and customer service 
activities are housed in a single facility in Urbana, Illinois. Two subjects were selected from each 
of the following areas: (a) design engineering, (b) production testing, and (c) customer or field 
service, based on supervisor nomination. 

Apparatus 

The equipment used for this investigation was an aircraft electrical system simulator board 
designed to provide realistic troubleshooting experiences and practical examinations at the Institute 
of Aviation at the University of Illinois at Urbana-Champaign. The selected system, Electrical 
Systems Simulator Board B, contained ten specific discrete subsystems which represent realistic 
aircraft electrical systems. Components included circuit breakers, switches, relays, terminal strips, 
conductors, and major functional system components such as a rotating beacon, power inverter, 
landing lights, control motors, and a fuel pump. 
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Task selection 

The process of task selection for this investigation was based on specific criteria to ensure a 
representation of specific populations of tasks. Task selection was also limited to problems that 
were not likely to result in immediate solution based on experience or cursory observation. 
Additionally, each problem was inserted so that it could be specifically identified A pilot test of 
the entire fault set was accomplished prior to data collection* Three faults were inserted which 
represented a range of contextual relevance* The first fault was a concealed piece of transparent 
tape covering the point contact of the power path to the lamps within the rotating beacon. The 
second fault was a mis-wired microswitch in the down Gear Indicator circuit The final fault was 
an incorrectly rated circuit breaker in the Inverter circuit and the circuit breakers value was altered 
on the schematic. The under-rated circuit breaker was a design flaw. The mis-wired microswitch 
simulated a production error, and the beacon contact simulated normal wear during operation, a 
repair fault. 

Data analysis 

The framework provided by the Technical Troubleshooting Model resulted in four linked 
reductionary levels of troubleshooting protocol analysis: (a) global performance, (b) fault isolation 
scenarios, (c) hypothesis generation/evaluation episodes, and (d) information 
acquisition/interpretation efforts. Following the coding and segmenting of verbal protocols 
(Ericsson & Simon, 1984) the protocols were divided into separate problem scenarios and general 
search areas not related to the three tasks. These data formed the basis for Global Pei* mance 
Matrices (See Figure 2) tii^r deluded the sequence of activities, subsystem* examined, time 
devoted to each activity, information acquisition efforts within the activity, results of the activity, 
and researcher comments for clarification. Following the division of the protocols into general and 
fault specific sequences the fault scenarios were further divided into episodic events and plotted on 
an Episodic Performance Matrix (see Figure 3). An episode is defined as a complex pass through 
the Technical Troubleshooting Model (Johnson, 1987, 1989). Each episode consisted of (a) the 
selection of at least one potential hypothesis, (b) the acquisition of information for the purpose of 
evaluating the hypothesis, (c) the interpretation of the acquired information for the purpose of 
evaluating the hypothesis, and (d) a decision to accept or reject the hypothesis* 



Insert Figures 2 and 3 about here 
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All sensory, test equipment based, or physically manipulative information acquisition 
efforts were analyzed for type, interpretation, self-correction of errors in interpretation, and 
redundancy. Fault specific efforts were plotted on functional circuit maps for each problem 
representing the order and problem space relevance of those eiforts (see Figure 4). Two additional 
analyses were accomplished First, specific subject fault activities for each task were represented 
in Episodic Performance Profile Graphs (See Figure 5). These graphs indicate interpretation of 
hypotheses, sequence and level of hypotheses, and problem space relationship of each hypothesis 
as well as the resultant solution related conclusions. Second, subject strategies for each fault 
scenario were placed in table form. The episodic matrices, categorical analyses, graphic 
representations, and strategy tables were compared and analyzed for commonalties and differences 
between subjects and for relationships to group and individual performance. 



Insert Figures 4 and 5 about here 



During the troubleshooting task, additional observation data was recorded by the researcher 
as well as the time devoted to each scenario. Observation data provided a source of clarification for 
the verbal protocol data, and a guide for recounting the activity (Cooke, 1988; Rasmussen & 
Jensen, 1974). An in-depth interview was conducted immediately after the troubleshooting 
performance task. The interview data was used to support and clarify the other data sources and 
generate specific contextual goal structuies and problem representation relationships. Near the end 
of the interview each subject was asked to place a circle around their position at Frasca on the 
Contextual Problem Solving Continuum for electronics troubleshooting. On a second graph they 
were instructed to place the circle around the area they were an expert in, or felt the most 
competent. On a third graph they were asked to place the origination of the faults encountered 
during the troubleshooting performance activity as a representation of the global problem space 
associated by the subject with each specific fault This data was compared by subject relative to 
group membership, reported expertise, and problem solution. 

Results 

The contexts under study represented three distinct task environments, defined by job 
related responsibilities. It appeared that beyond knowing where to start in the process, actual 
troubleshooting process differences were few. Generally, all subjects were able to perform in a 
manner consistent with expertise as defined by solving the problem representative of their 
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respective contexts. There appeared to be little overall difference in the ability of the 
troubleshooters to generate and evaluate hypotheses, acquire and interpret information, or select an 
appropriate problem spaces beyond the contextual reference. Difficulties with interpretation of 
information or hypothesis evaluation were generally overcome through self correction and 
continued data gathering. In essence subjects from all groups were able to demonstrate high levels 
of skill associated with expertise in electronics troubleshooting within problems representative of 
their typical job contexts and experience. 

Problem Solutions 

Correct solutions occurred in nine of eighteen potential scenarios (see Table 1), All correct 
solutions were in reported areas of expertise, and two-thirds of the incorrect or inconclusive 
solutions were outside of subjects reported contexts of expertise. When asked to represent their 
areas of expertise on the Contextual Problem Solving Continuum only two subjects indicated 
expertise beyond their specific positions. Subject Rl who had worked in production prior to field 
service (repair) claimed a dual expertise in those areas, and subject D2 included the entire range of 
contexts in his response. During the structured interview subjects were again asked to indicate 
what additional areas they could work in, interview responses and graphic representations are 
summarized in Table 2. 



Insert Tables 1 and 2 about here 



There were traces of context effects evident in the subject's troubleshooting performance. 
The design problem was the clearest example. The two design subjects were the only individuals 
able to solve the design fault Even when the other subjects had acquired the current ratings from 
both the Inverter and circuit breaker, they failed to consider the circuit's design. It is unlikely that 
the other subjects did not have the knowledge necessary to solve the problem as it did not require 
circuit analysis, formula manipulation, or theoretical abstraction, only a simple comparison of the 
two component values. It appeared that the other subjects relied on an inappropriate "frame of 
reference" or initial problem representation that they did not question. 

The production group members only arrived at one correct solution, to the production 
problem. That area appeared to be common to the other groups in reported expertise. This may 
have been due to both previous experience in that context, and some degree of common activity 
associated with production tasks. Both designers reported checking wiring in prototypes as 
common design activities while repair personnel had to verify installations or component 
replacements. One production subject did not recognize the problem. This appeared to be due to a 
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transfer of technique from his routine activities that did not include obvious symptomatic 
conditions. That subject worked on radio units and relied on oscilloscope tests to provide a 
"window" for circuit symptoms. Additionally, that subject appeared to be completely dependent 
on the schematic, and in fact stated that without a schematic he could not troubleshoot A design 
subject also failed to solve the production fault, concluding that the Gear Indicator instrument had 
failed. 

The repair subjects were able to solve both the production and repair problems, although 
they reported previous experience in both contexts. A design group member was the only other 
subject able to solve the repair fault He also reported experience in field service activities. Two 
subjects, one design and one production, failed to pursue a maintenance orientation to the level 
necessary to effect a repair. Both subjects, although in a maintenance frame, would have replaced 
an entire unit for an insignificant internal fault 

Subjects with multiple context experience appeared to have little difficulty in solving 
problems from different contexts. Subject Dl, the only subject able to solve all three faults, stated 
that he would have been more efficient on the design problem but initially he "wasn't in my design 
mode." This was the only direct reference to changing frames of reference during the 
troubleshooting activity. The awareness of the appropriate problem representation allowed the 
subject to "shift" to a relevant problem space and consider a related set of potential hypotheses. 

Hypothesis Generation and Evaluation 

Global hypothesis generation and evaluation data for each subject is presented in Table ?. 
The evaluation of hypotheses resulted in three outcomes (a) correct interpretations (b) incorrect 
interpretations, and (c) incomplete interpretations, or failure to complete the evaluation due to 
abandoning the hypothesis or reaching an impasse during the process. With the exception of 
Subject Dl who correctly solved every fault, and Subject P2 who did not solve any faults, a 
difference can be observed between hypothesis evaluation within correct solutions and overall 
evaluation efforts. Generally little difference can be attributed to group membership for the ability 
to evaluate hypotheses given the demonstrated evaluation accuracy within correct solutions. 
Subjects demonstrated a high level of correct hypotheses evaluation within the problems they 
correctly solved, consistently exhibiting characteristics associated with expertise for context 
representative problems. 



Insert Table 3 about here 
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Information Acquisition and Interpretation 

Two subjects (PI and R2) correctly interpreted all of the information they acquired within 
the problems they solved correctly (see Table 4). When adjusted for self-correction of 
interpretation mistakes, however, all subjects addeved completely correct interpretations for the 
problems they successfully solved. All subjects, regardless of group membership, were capable of 
interpreting the information sought, particularly within problems representative of their contextual 
focus. Total correct interpretations for all problems attempted ranged from 75% for Subject P2, to 
100% for both subjects Dl and D2. With The exception of Subject P2, all subjects attained a 
general interpretation rate, after self-corrections, of over 90%. 



Insert Table 4 about here 



Problem Space Representations 

The range of acquisition efforts that were conducted in the appropriate subsystem problem 
space does not appear to reflect contextual group membership (see Table 4). This problem space 
does not reflect the depth of context within the problem space, but instead only the likelihood of 
fault potential within the component set. The range of .50 to .81 for all problems, or the correct 
solution range of .40 to .93, is more reflective of the problem type than subjects' general ability. 
The schematic diagram for Board B did not include detailed information concerning voltage paths, 
or internal operation for all circuits. The subjects had no previous experience with the system, and 
therefore often had to learn the circuitry while diagnosing the faults. 

An overall indication of problem space depth can be developed from the subjects' graphic 
representations on the Contextual Problem Solving Continuums. Subjects were asked to place 
each problem in an appropriate range, representing an overall problem space characterization. 
Without exception each one of the correct component or device level solutions (n=l 1) were plotted 
in the correct corresponding area on the Contextual Problem Solving Continuums. Four of the six 
incorrect, or inconclusive solutions were placed in an incorrect area and two were appropriately 
placed The solutions that were placed outside of the context problem space are indicative of the 
use of an inappropriate reference frame. The representations demonstrate that the subjects 
recognized the problems as characteristic of the specific contextual areas, and that the correct 
representation was consistent with achieving a correct solution. 
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Troubleshooting Strategies 

Effective troubleshooting strategy use, including the ability to use multiple strategies when 
necessary, is another essential component of skilled troubleshooting (Johnson 1991; Johnson et 
ah, 1992). The primary strategies used by each subject are presented in Table 5. The subjects 
selected troubleshooting strategies without any evidence of limited strategy performance during 
fault isolation. All subjects demonstrated an ability to use multiple strategic approaches including 
functional evaluation and half-split searches. As with the previous indicators of expertise, little 
difference exists between groups in their ability to use multiple strategies in a fashion consistent 
with expertise. 



Insert Table 5 about here 



Discussion 

The findings of this study must be carefully interpreted, specifically with regard to the 
context of the study itself. The primary purpose of this study was to explore the relationship of 
context cuid electronics troubleshooting performance among experts, in context related tasks. The 
small sample size, while providing implications, precludes statistical analysis treatments and any 
inference or generalizability associated with them. Additionally, the population from which the 
subjects were selected, has a specific set of non-generalizable characteristics. The definition of the 
three context groups is dependent upon the activities they engage in, and mission of the industry or 
institution in which they perform troubleshooting. Given the limitations inherent in this study the 
results cannot be generalized to a larger population other than the limited sample of aircraft 
simulator electronics experts selected The results can be used, however, to guide additional 
research efforts both in electronics troubleshooting expertise, and other domains of expertise. 

Implications for Instruction 

The general process skills associated with troubleshooting, and documented in this as well 
as other research efforts (Johnson, 1988; Johnson et al., 1992; Lesgold et al., 1986; Rasmussen & 
Jensen, 1974), must be included as an integral part of electronics instruction at all levels. All three 
groups of electronic experts engage routinely in troubleshooting activities that follow much the 
same process. The instruction of troubleshooting process at any level should include the elements 
of problem representation, hypothesis generation and evaluation, strategy selection, and 
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verification of results. Additionally, the awareness of contextual references and the associated 
baseline assumptions they provide should be an explicit element of instruction, particularly when 
the goal of instruction is flexible expertise or general preparation for a non-specific role in the 
workplace. 

An awareness of the effects of context, and the definitions of expertise resulting from 
context, can assist instructional designers and curriculum planners in providing relevant content 
from an appropriate reference* Electronics instruction aimed at maintenance technicians, but taught 
from a design point of view will appear to have little relevance to the program graduate or the 
employer. In addition to electronics instruction, curriculum planning for any multiple context 
endeavor should review the role of context- and particularly in instructional design guided by 
subject matter experts, ensure fidelity of instructional and target contexts. 

Implications for the Study of Expertise 

The role of context has important implications for the general study of expertise. A 
thorough examination of context could serve to more accurately define the limits of expertise and 
account for the irregularities often encountered in expert performance. Additionally, the systematic 
investigation of context can aid in the selection of an appropriate population of tasks for problem 
solving research. 

In previous studies were novices and experts, or multiple groups of experts, have been 
compared little attention has been paid to context effects. Particularly in studies that compare 
groups, contextual references should be controlled for and acknowledged. Characterization of 
experts on a skill continuum with contextually disparate subjects may have little validity. Studies 
that have used a single subject matter expert, from a disparate context, may also lack foundational 
validity due to context effects. 

The process of developing problem categories to guide expertise research efforts should 
account for variations in normally encountered task environments. Although the Contextual 
Problem Solving Continuum used in this study provides one level of definition it is likely that 
increasingly exact definitions can be developed with further research. Certainly, the tasks provided 
to experts by researchers, and the contexts they represent, are critical factors in studies of 
expertise. The selection of an inappropriate research context, or representative sample of tasks, 
may well provide an erroneous outcome. 

Recommendations for Further Research 

The most important result of this study may be the implications for further research 
discovered during this process. Certainly more research is needed that investigates the role of 
context in troubleshooting expertise, and the knowledge bases needed to support troubleshooting 
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activities. Additional sources of study could include the manipulation of contextual reference data, 
differing problem presentations over multiple trials, and the development or use of other data 
collection avenues. 

In-depth study of both design and production troubleshooting is needed. These studies 
could develop a body of knowledge for comparison to the existing maintenance process 
information. Multiple trials based on varying types of problems within each context is needed as 
well. An interesting variation would be providing references to u:e subjects when they exhaust 
their process, or as a function of multiple group experimental designs. Another promising 
approach may be the collection of verbal protocols in the natural environment over multiple 
occasions of work related troubleshooting- Constructing varying levels of context around 
problems could also provide useful insight These presentations could range from written 
problems framed in a context "story problem" to introducing a problem set as representative of a 
context, or even realistic simulations of the various environments. 

Care must be taken within studies that investigate context effects to limit the interference of 
research presentation. The smallest amouni of problem introduction could influence the problem 
representations developed by the subjects. In order to gain additional insight into contexts, and 
frames of reference, it may be beneficial to stop the subjects before they attempt to gain any 
additional information through tests or system manipulations. A recursive interview at this point 
could provide some indication of initial context definition and subjects' frames of reference. 
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Subject PI Gear Indicator Episodic Performance Graph 
Correct Component Level Solution 
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Problem Space Relationship: 

In Problem Space (1) System, (2) Subsystem. (3) Device. (4) Component 

Out of Problem Space (-1) System, (-2) Subsystem. (-3) Device, (-4) Component 



Figure 5 Episodic Per formance Profile Graph 
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Table 1 

Reported Context Expertise 
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Context Subjects Reporting Expertise 

Design D1,D2 

Production PI, P2, Dl, D2, Rl, R2 

Repair Rl, R2, Dl, D2 



Table 2 

Correct Problem Solutions 



Problem Representative Context Correct Subjects 

Inverter Design D1,D2 

Gear Indicator Production P 1 , D 1 , R 1 , R2 

Rotating Beacon Repair Rl, R2, Dl 



Table 3 

Hypotheses Evaluation 



Context and Expertise 
25 



Subject All Problems Attempted Correct Solutions Only 





Correct 


Incorrect 


Incomplete 


Correct 


Incorrect 


Incomplete 


Dl 


.86 


.07 


.07 


.86 


.07 


.07 


D2 


.50 


.50 




1.0 






PI 


.83 


.17 




1.0 






P2 


.33 


.50 


.17 








Rl 


.57 


.29 


.14 


.67 


.22 


.11 


R2 


.68 


.05 


.27 


1.0 







Table 4 

Information Acquisitions 
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Subject 


Correct 
Interpretation 


Redundant 
Acquisitions 


Recoverv 
From Errors 


Total C^cwvf^c^t 

Interpretations 


rruuicui opdcc 
Accuracy 








All PmhIfMYw 






Dl 


.99 


.10 


.10 


1.0 


.73 


D2 


.75 


.10 


.25 


1.0 


.50 


PI 


.94 


06 


03 


97 


.OA 


P2 


.55 


.35 


.20 


75 


. / o 


Rl 


.89 


.14 


07 


96 


74 


R2 


.92 


.19 


.01 


.93 


77 








Correct Solutions 






Dl 


.99 


.01 


.01 


1.0 


.73 


D2 


.57 


0 


.43 


1.0 


.86 


PI 


1.0 


.10 




1.0 


.40 


P2 












Rl 


.90 


0 


.10 


1.0 


.93 


R2 


1.0 


.05 




1.0 


.76 



Table 5 

Troubleshooting Strategies 
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Subject Beacon Gear Indicator Inverter 



Dl 


Functional evaluation 
Visual inspection 
Half-Split voltage 


Functional evaluation 
Visual inspection 


Functional evaluation 
Visual inspection 
Half-Split voltage 


D2 


Functional evaluation 
Reverse voltage 


Half-Split voltage 


Functional evaluation 
Half-Split voltage 


PI 


Functional evaluation 
Visual inspection 
Physical half-split 
Reverse voltage 


Functional evaluation 
Operational comparison 


Functional evaluation 
Visual inspection 
Physical half-split 


P2 


Trial and error 
Forward voltage 


None 


Functional evaluation 
Physical half-split 
Half-Split voltage 


Rl 


Functional evaluation 
Experimentation 
Physical half-split 
Visual inspection 
Reverse voltage 


Functional evaluation 
Experimentation 


Functional evaluation 
Experimentation 
Physical half-split 
Visual inspection 
Half-Split voltage 
Exhaustive search 


R2 


Functional evaluation 
Experimentation 
Reverse voltage 


Functional evaluation 
Experimentation 
Reverse voltage 


Functional evaluation 
Physical half-split 
Visual inspection 
Forward continuity 
Exhaustive search 
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